Mixed Fantasy: An Integrated System for Delivering MR Experiences

This paper describes the underlying science and technologies employed in a system we have developed for creating and delivering multimodal Mixed Realities. We also present several aspects of a number of experiences that we have developed and delivered, when such aspects help understanding of algorithmic requirements and design decisions. The technical contributions include a unique application of retro-reflective material that results in nearly perfect chroma-key, even in the presence of changing illumination, and a component-based approach to scenario delivery. The user interface contribution is the integration of 3-D audio, hypersonic sound and special effects with the visuals typical of an MR experience.